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Description 

BACKGROUND OF THE INVENTION 

5 1. Field of the Invention 

[0001] The present invention relates to a stereoscopic image pickup apparatus for recording and displaying a ster- 
eoscopic image of an object. 

10 2. Description of the Related Art 

[0002] An example of a prior art stereoscopic image pickup apparatus is shown in Figure 10. 

[0003] The apparatus shown comprises two ordinary cameras mounted on two separate universal heads, the two 

cameras corresponding to the left and right eyes of a human. The cameras are secured to rotation controllers 3 and 

is 4, respectively, so that they are rotatable in a horizontal plane. The spacing between the two cameras is adjustable on 
a camera spacing controller 5. The entire camera construction is supported on a tripod 6. Top plan views of this prior 
art stereoscopic image pickup apparatus are given in Figures 11(a) and 11(b): (a) shows the apparatus in a parallel 
shooting position where the optic axes of the cameras are set parallel to each other, and (b) illustrates a converging 
shooting position where the optic axes of the cameras are tilted so that they intersect each other. The rotation controllers 

20 3 and 4 are operated to rotate the respective cameras and set them in the parallel shooting or the converging shooting 
position. The camera spacing is adjustable by varying Wc. In shooting an object in three dimensions, the camera 
spacing Wc and the distance dx to the converging point greatly affect the quality of the resulting stereoscopic image. 
The camera spacing Wc affects the size of the object to be displayed as well as the amount of the horizontal parallax 
to be reproduced. The distance dx to the converging point has a significant effect on the distortion of the recorded 

25 image as well as on the amount of the horizontal parallax (which affects the binocular fusing range of the viewer). In 
the prior art stereoscopic image pickup apparatus, the camera spacing Wc and the distance dx to the converging point 
are set by an operator based on his experience so that the recorded image can be perceived properly when viewed 
by a viewer. Here, "perceived properly' essentially means bringing the horizontal parallax of the stereoscopic image 
within the fusing range of the viewer's eyes when the image is viewed by the viewer. The fusing range will be explained 

30 jn further detail in the preferred embodiments of the present invention. Figure 12 is a schematic diagram of a prior art 
stereoscopic image display apparatus for reproducing a stereoscopic image signal recorded by the stereoscopic image 
pickup apparatus. In the display apparatus, two separate images, one for each eye, recorded by the stereoscopic 
image pickup apparatus are reproduced by projectors 9 and 10, are polarized through two polarizers 11 oriented per- 
pendicular to each other, and are focused onto a screen 12. When these images are viewed by a viewer wearing a 

35 pair of polarizing glasses 1 3, the right-eye image is reflected only into the viewer's right eye and the left-eye image 
only into the viewer's left eye. 

[0004] Figure 13 shows the screen of the stereoscopic image display apparatus as viewed from above. If IL and IR 
represent the left- and right-eye spot images obtained by the stereoscopic image pickup apparatus, a viewer with an 
interocular distance of We perceives the object virtually lying at the intersection C1 between the visual axes. Thus, the 

40 viewer can perceive the depth of the displayed image by binocular stereoscopic vision. In another prior art stereoscopic 
image display method, as shown in Figure 14, images displayed on two CRTs with crossed polarizers are combined 
by a half-silvered mirror without using a screen, and the combined image is viewed through glasses with the corre- 
sponding polarizers. In any case, the principle of making the viewer perceive the depth of an image is the same. 
[0005] Next, the perception of depth will be described in further detail with reference to Figure 15. This diagram 

45 explains how a viewer perceives image depth based on the parallax between his left and right eyes. In the diagram, 
A, B, and P represent objects (points), CL is the center of rotation of the viewer's left eye 26, CR is the center of rotation 
of the viewer's right eye 27, and FL and FR are foveae of the viewer's left and right eyes, respectively (the fovea is the 
region in the retina where there is the greatest acuity of vision; when a human views an object, an image of the object 
is focused onto the fovea ). Now suppose that the viewer focuses his eyes on the point R In this case, the left and right 

50 eyeballs move so that images of the point P are focused on the respective foveae , FL and FR. At the same time, 
images of the point A situated nearer to the eyes than the point P are focused on AL and AR on the retinas of the left 
and right eyes. Similarly, images of the point B farther from the point P are focused on BL and BR on the retinas of the 
left and right eyes. At this time, the distance ALBL between the images of the point A and point B formed on the retina 
of one eye is different from the corresponding distance ARBR in the other eye. This difference in distance corresponds 

55 to the distance between the point A and point B in the direction of depth. In terms of angle, the binocular parallax 8BA 
between the point B and point A is expressed as yBA = 7B - yA , the amount of which corresponds to the distance 
between the point A and point B in the direction of depth. A human calculates this difference in his brain to perceive 
the depthwise position of an object, i.e., LAB in Figure 1 5. As the distance to an object increases, a human's perception 
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of the absolute distance LA, LB to the point A, B becomes less sensitive while his perception of the relative distance 
LAB becomes more sensitive. 

[0006] However in the prior art construction described above, a viewer perceives the object lying at position C1 
when the actual images are focused on the display screen 12. That is, the actually displayed images and the image 

s as viewed by the viewer are at different fixating positions of the eyes. This gives rise to a conflict between the conver- 
gence point of the visual axes of the viewer's eyes (both eyes turn inward with the eyeballs directed to the nearer 
position C1) and the accomodation point of the eyes (fixating of the eyes). If the amount of this conflict becomes too 
large, a human becomes unable to perceive the left and right images as a single image, and perceives them as a 
double image. Even if the image does not appear double, the resulting stereoscopic image may appear very unnatural 

io and may make the viewer feel sick. 

[0007] To describe in further detail, in Figure 15 where the viewer focuses his eyes on the point P, a human can 
calculate the depth when the amount of the binocular parallax 5 is within a range of ±0.5°; if the depth wise distance 
becomes larger (the absolute value of the binocular parallax 5 becomes larger), the sense of depthwise distance de- 
creases, and if the distance increases further, the object will appear double, making it difficult to perceive the depth 

15 and the image very unpleasant to view. Here, suppose that the points A, B, and P in Figure 15 are displayed on the 
stereoscopic image display apparatus of Figure 1 4. As the position of the reproduced stereoscopic image moves farther 
away from the position 28 of the CRT surface where the images are displayed, the displacement between the fixating 
position (CRT surface) of the eye lens and the depthwise position of the presented image becomes larger. Human 
vision has a threshold of tolerance for the displacement between the CRT surface and the stereoscopic image display 

20 position. If the displacement becomes larger than this threshold, images of the intended object cannot be focused on 
the foveae of both eyes, and the viewer perceives them as a double image. 

[0008] Considering the above facts, it is clear that a human's depth perception by binocular stereoscopic vision is 
limited to within a certain range centering around the fixating point P To perceive a broader three-dimensional world, 
a human moves the fixating point P in depthwise direction, perceives three-dimensional geometries around respective 

25 fixating points, and merges them to perceive the entire perspective. However, with the prior art stereoscopic image 
display apparatus, since there are tolerable limits to the displacement between the fixating position (CRT surface) of 
the eye lens and the depthwise position of the presented image, it is not possible to observe the entire space around 
the object. Furthermore, since there always is a displacement between the fixating position (CRT surface) of the eye 
lens and the depthwise position of the presented image, it is difficult to move the fixating point quickly. These factors 

30 combined, the prior art stereoscopic image display apparatus has not been able to present images that are easy on 
the eyes and that have good visibility. 

[0009] To compensate for these shortcomings, in operating the prior art stereoscopic image pickup apparatus the 
operator had to measure the shooting distance in the field and determine Wc and dx in Figure 11 by accounting for 
the size of the stereoscopic image display apparatus used and the viewing distance from the audience, thereby avoiding 

35 the above-stated situation as far as possible. However, the calculations were not accurate since they were based on 
rough measurement of the shooting distance or on the experience of the operator, and camera parameters were set 
focusing on a single object so that the object could be viewed in three dimensions with both eyes without strain. Further, 
where the depth to be reproduced varies in the time domain, such as the object moving in depthwise direction, it was 
not possible to determine We and dx optimally. ' 

40 [001 0] EP-A-0 279 092 describes a pictu re processing system for improving the quality and viewing comfort of three- 
dimensional movies. The system utilises two stereoscopically related pictures which contain object images. The object 
images in one or both pictures are repositioned such that the positions of corresponding object images coincide ade- 
quately for comfortable viewing. 

[001 1] US-A-5, 175,61 6 describes a stereoscopic video-graphic co-ordinate specification system which superimpos- 
es es a 3D pointer on the picked up stereo images. The user views the pointer and the stereo images and moves the 
pointer to a position which the viewer wishes to look at. The converging angle of the camera is then controlled so as 
to make the parallax of the 3D pointer fall within the fusional range of the user. 

[001 2] JP-A-60236394 describes a stereoscopic television which automatically controls the converging angle (called: 
"parallax angle") of the stereo camera so as to have a certain relation to the converging angle of the viewer. 
so [0013] It is the object of the invention to provide a stereoscopic image pickup apparatus in which controlling the 
converging angle is achieved automatically on the basis of image information. 
[0014] This object is solved by the subject matters of independent claims 1 and 3. 
[0015] Preferred embodiments of the invention are defined by the dependent claims. 

[001 6] With the above construction of the present invention, the binocular parallax of a captured image is detected, 
55 based on which the depthwise position of an object at the nearest point to the viewer is calculated, and the converging 
angle moving mechanism is controlled so that the nearest point comes within the fusional range of the viewer viewing 
the reproduced image; furthermore, from the binocular parallax of the image, an optimum fixating point is calculated 
that enables the viewer to perceive the stereoscopic depth of the object over the widest possible range, and control is 
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performed so that the fixating point is reproduced at the surface of the stereoscopic image display or at a designated 
distance from the surface. 

BRIEF DESCRIPTION OF THE DRAWINGS 

5 

[0017] 

Figure 1 is a diagram showing the configuration of a stereoscopic image pickup apparatus according to a first 
embodiment of the invention; 

io Figure 2 is a set of diagrams showing an imaging coordinate system and a display coordinate system, respectively, 

for a stereoscopic image; 

Figure 3 is a diagram showing allowable ranges of a human's adjusting mechanism and converging mechanism; 
Figure 4 is a diagram showing a method of calculating the parallax; 

Figure 5 is a diagram showing a binocular fusing range in a stereoscopic image display apparatus (at viewing 
is distance > 71 cm); 

Figure 6(a) is a diagram illustrating the definitions of distance dx to the converging point and angle 9 of optic axis 
rotation, and 6(b) is a diagram plotting distance dx to the converging point as a function of angle 9 of optic axis 
rotation; 

Figure 7 is a diagram showing the configuration of a stereoscopic image pickup apparatus according to a second 
20 embodiment of the invention; 

Figure 8 is a diagram showing the configuration of a stereoscopic image pickup apparatus according to a third 
embodiment of the invention; 

Figure 9 is a diagram showing a binocular fusing range in a stereoscopic image display apparatus (at viewing 
distance < 71 cm); 

25 Figure 10 is a schematic diagram of a prior art stereoscopic image pickup apparatus; 

Figure 11 is a set of diagrams showing stereoscopic image shooting methods (parallel shooting and converging 
shooting); 

Figure 1 2 is a schematic diagram of a stereoscopic image display apparatus; 
Figure 1 3 is a diagram showing a stereoscopic image display position; 
30 Figure 1 4 is an external view of a prior art stereoscopic image display apparatus; 

Figure 15 is a diagram explaining how a viewer perceives depth based on binocular parallax; 

Figure 16 is a diagram showing the configuration of a stereoscopic image pickup apparatus according to a fourth 

embodiment of the invention; 

Figure 17 is a diagram for explaining a technique for obtaining weighted average of parallax; 
3S Figure 18 is a diagram showing the operation of a parallax controller 12; . . 

Figure 19 is a diagram showing the configuration of a stereoscopic image pickup apparatus according to a fifth 
embodiment of the invention; 

Figure 20 is a diagram showing the operation of a line-of-vision sensor; 

Figure 21 is a diagram showing a method of calculating the coordinates of an fixating point in a displayed image; 
40 Figure 22 is a diagram showing a method of calculating the position of the fixating point; 

Figure 23 is a diagram showing a method of calculating the control amount for horizontal read timing of left and 
right images; 

Figure 24 is a diagram showing the configuration of a stereoscopic image pickup apparatus according to a sixth 
embodiment of the invention; and 
45 Figure 25 is a diagram showing the configuration of a stereoscopic image pickup apparatus according to a seventh 

embodiment of the invention. ^ 

DESCRIPTION OF THE PREFERRED EMBODIMENTS 

so [0018] Figure 1 is a diagram showing the configuration of an image pickup section in a stereoscopic image pickup 
apparatus according to a first embodiment of the present invention. In Figure 1 , the numerals 1 and 2 are lenses, 3a 
and 3b are camera bodies, 4a and 4b are rotation controllers, and 5 is a camera spacing controller. These components 
are the same as those used in the prior art. The numerals 14a and 14b are signal processors each for converting the 
camera signal into the luminance signal, 1 5 is a parallax calculator for calculating horizontal parallax from left and right 

55 images, 16 is a parallax processor for detecting the smallest parallax, 17 is a depth display position calculator for 
calculating the spatial position of an object nearest to the viewer, 18 is a fusing range verifier for verifying whether the 
spatial position of the object nearest to the viewer is within the fusing range of the viewer's eyes, and 1 9 is a converging 
angle controller for controlling the rotation controllers 4a, 4b so that the spatial position of the object nearest to the 
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viewer cornes within the fusing range of the viewer's eyes. 

[001 9] Before describing the operation of the thus constructed stereoscopic image pickup device and display section 
of the present embodiment, we wit! first describe the modeling theory of the image pickup system and display system 
and the characteristics of a viewer's binocular fusing range. 

s [0020] First, models of the image pickup system and display system will be described with reference to Figure 2. 
Figure 2(a) shows a model of the image pickup system. The image pickup cameras are arranged symmetric to each 
other with their optic axes directed to a converging point F (0, dx, 0). In the coordinate system shown here, the center- 
point of the line joining the two cameras is the origin, the two cameras are arranged along the x-axis, the direction of 
depth is plotted along the y-axis, and the direction of height along the z-axis. The left and right lenses are positioned 

10 at (-Wc, 0, 0) and (Wc, 0, 0), respectively, on the x-axis. The focal length of each lens with respect to the imaging 
surface (the position of the imaging device or film) 7, 8 is denoted by f. In this arrangement, when dx = <», the optic 
axes of the two cameras are parallel to each other (shown by dashed lines in Figure 2(a)). Figure 2(b) shows a model 
of the display system (a model of a stereoscopic image display apparatus). In the display coordinates system, the 
origin represents the centerpoint between the two eyes of the viewer, the line joining the centers of the two eyes is 

is taken as the x-axis, the direction of depth is plotted along the y-axis, and the direction of height along the z-axis. In 
this geometry, let the distance from the viewer to the stereoscopic image display screen be denoted by ds, the positions 
of the left and right eyes be represented by coordinates (-We, 0, 0) and (We, 0, 0), respectively, and the positions of 
reproduced spot images be designated Pr for the right eye and PI for the left eye. Then, the viewer will perceive the 
spot image lying at the point of intersection, P, of the line joining Pr and (We, 0, 0) and the line joining PI and (-We, 0, 

20 o). Generally, in a stereoscopic image display apparatus having a flat screen such as shown in Figure 2(b), when dx 
= , and when the position at which a point of infinity was captured (the center position of the captured image) is 
displayed at NIL (-We, ds, 0) for the left image and at NR (We, ds, 0) for the right image, then the x direction of the 
imaging space, when reproduced, is free from distortion. Further, the y direction of the imaging space, when reproduced, 
is free from distortion when the following equation holds. 

25 

[Mathematical 1] ds=M * f 

where M is the ratio of the image on the imaging surface 7, 8 to the size of the image on the screen 12 (provided that 

30 the center of the viewer is positioned on the y-axis). 

[0021] In this case, when viewing a reproduced image of a point of infinity, the visual axes of both eyes are parallel 
to each other as shown by the dashed lines in Figure 2(b). Since the object is positioned somewhere between the point 
of infinity and the distance of 0, the visual axes of the viewer are either directed in parallel or turned inward, but are 
not turned outward. Since human eyes cannot be turned outward of the parallel lines at the same time, the above 

35 provides the condition that achieves the widest possible range for the reproduction of a stereoscopic image. 

[0022] Of the above distortion-free reproduction conditions, in cases where the condition of dx = <» in Figure 2(a) is 
not satisfied, that is, in cases where dx takes a finite value, the resulting stereoscopic image always suffers spatial 
distortion. Therefore, the condition of dx = °° should be observed if at all possible, and only when the resulting stere- 
oscopic image does not lie within the fusing range of the viewer's eyes, dx is varied so that the viewer can fuse the 

40 images into a single image. In this case, the value of dx should be set as large as possible to minimize the distortion 
of the reproduced stereoscopic image. How dx is actually controlled will be described later. At the stereoscopic image 
reproduction side, images of a point of infinity are reproduced at NR and NL shown in Figure 2, for maximum repro- 
duction range of the stereoscopic image. To achieve this, since the horizontal coordinate of the point of infinity on the 
imaging surface 7, 8 in Figure 2(a) is fx Wc / dx, AS is set so that the following equation holds, displaying the left and 

45 right images by shifting the right image by AS horizontally to the right and the left image by AS horizontally to the left. 

[Mathematical 2] We = M ' f dx Wc + AS 

so [0023] Next, the binocular fusing characteristics of a human will be described. In the present invention, the parameter 
dx in the image pickup apparatus is controlled so that when a stereoscopic image is displayed by the stereoscopic 
display apparatus, the stereoscopic image can be displayed within the range in which the viewer can fuse the left and 
right images into a signal image. It is therefore necessary that the condition that enables the viewer to achieve binocular 
fusion be first determined in the geometry shown in Figure 2(b). In Figure 2(b), suppose, for example, that the viewer 

55 js viewing the point P. In this case, the actual images are focused on the display screen 12. However, the visual axes 
of both eyes are directed to the point P Crossing both eyes inward to view a near object is known as convergence 
eyemovement, and the point P is called the converging point. The convergence eyemovement is controlled using the 
eye's focus information as well as the information concerning the projected position of the point P on the retina. 
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[0024] In this case, since there is a conflict between the tocus information and the retina information, strain is imposed 
on the viewer's eyes. When the conflict remains within a limited range, the viewer can tolerate the strain, but when the 
conflict becomes large, the viewer becomes unable to fuse the left and right images into a single image, and eventually 
perceives them as a double image. Even if the image does not appear double, it imposes strain on human's visual 

5 system, leading to fatigue. 

[0025] Allowable ranges for the fixating position of the eye (hereinafter called the accomodation point) and the position 
of the converging point (hereinafter called the convergence) based on the projected positions on the retinas were 
measured. The results are shown in Figure 3. The convergence plotted along the abscissa is represented by the 
distance between the viewer and the intersection of the visual axes of both eyes, and the accomodation point plotted 

10 along the ordinate is represented by the distance (measured in diopter, the reciprocal of the distance) between the 
viewer and the position on which the lenses of the eyeballs are focused. 

[0026] The straight line upward to the right at an angle of 45° represents the condition where the convergence co- 
incides with the accomodation point, the shaded area shows the allowable range within the depth of focus of the eye, 
and a indicates the fusional limits when an image is presented for 0.5 second. Within the region between the upper 
is and lower fusional limits, the viewer can fuse the stereoscopic images and no double image will occur. In shooting an 
object, therefore, control is performed so that dx in Figure 2 will have the largest possible value while satisfying the 
condition that enables all stereoscopic images to be reproduced within the fusional range. 

[0027] First, dx is initially set at <» (sufficiently large value). The parallax calculator 15 shown in Figure 1 calculates 
a parallax map (a three-dimensional map) from the left- and right-eye images output from the signal processors 14a 

20 and 1 4b. For calculation, there have been proposed a variety of methods, such as a correlation matching method that 
calculates the correlations of luminance patterns between left and right images, and an edge information matching 
method that involves matching the edge information between left and right images. The example hereinafter described 
employs the correlation matching method that calculates the correlations of luminance patterns. The operation of the 
parallax calculator 15 will be described in detail with reference to Figure 4. In Figure 4, consider left and right images 

25 each of N x M size. In the left image, consider a block window of n x n pixels (3x3 pixels in the figure). The same 
image as shown in this block window is located in the right image by using a window of the same size. At this time, 
the displacement between the left and right blocks is represented by a vector (Ax, Ay), whose horizontal component 
Ax indicates the binocular parallax of the left and right images at the center coordinate of the block window By hori- 
zontally shifting the reference block window in the left image in sequence across the entire screen, and finding the 

30 corresponding block position (representing the binocular parallax) in the right image for each shifted block position, a 
parallax map (showing depthwise distance at each position on the screen) can be obtained for the entire screen. The 
displacement between the left and right images at coordinates (x, y), that is, the binocular parallax (Ax, Ay), can be 
expressed as 



35 



40 



45 



[Mathematical 3] Ax=i , Ay=j for Min (Corr(iJ)} 

where 

[Mathematical 4] 

nxni 

Corr(i, ])= Y, GL(xk,yk)-GR(xk-i,yk-j) 

k=l 



In the expression shown in Mathematical 3, z means taking the sum of the absolute values by varying the coordinates 
50 xk, yk within the block window of n x n. N is the number of pixels in the block window. In the binocular parallax Ax, Ay, 
the component that directly indicates the depthwise position is Ax. With the left image as the reference, if the value of 
the binocular parallax is positive, the right image is positioned to the right, and the left image to the left, with respect 
to the reference image, the object lying farther than the depthwise position at which the binocular parallax is zero; on 
the other hand, if the value of the binocular parallax is negative, this indicates that the object is positioned nearer than 
55 the depthwise position with zero binocular parallax. 

[0028] Based on the thus obtained parallax map (showing parallax Ax calculated at each coordinate position on the 
entire screen), the parallax processor 16 extracts the smallest parallax value (the parallax of the object reproduced at 
the position nearest to the viewer) from the entire screen, the extracted value being denoted as Axmin. Next, based 
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45 



on Axmin, the depth display position calculator 17 calculates the depth reproduction position Ypmin of the stereoscopic 
image on the stereoscopic image presentation diagram shown in Figure 5, using the following equation. 

[Mathematical 5] 



2We-ds 



Ypmin = ^ ^ ^ VAI 

(max) -M-AXmin-2-AS-f-2We 
(max) 

*5 where Ypmin represents the y-coordinate of the distance from the origin of the display coordinate system to the position 
of the object in fixating. In Figure 5, the viewer's binocular fusing range shown in Figure 3 is superimposed on the 
display model of Figure 2(b). The shaded portion shows the binocular fusional region of the viewer. In the figure, F(ds) 
min corresponds to the lower curve in Figure 3. When screen viewing distance ds = 71 cm, then F(ds)min = 29 cm, 
which means that the viewer can fuse images within the range from infinity to 29 cm in distance. 

20 [0029] As can be seen from Figure 3, when the viewing distance is 71 cm or greater, the viewer can fuse images 
within the range from infinity to F(ds)min. When the viewing distance is smaller than 71 cm, the binocular fusing range 
is also limited by the upper curve in Figure 3, thus limiting the binocular fusing range to within the distance from F(ds) 
max to F(ds)min. The following description deals with a case where the viewing distance is 71 cm or greater. The fusing 
range verifier 18 judges whether Ypmin lies on the boundary of the binocular fusing range. 

25 [0030] The judgement is made by evaluating whether the following relation holds or not. 

[Mathematical 6] F(ds)min+AF ^ YPmin ^ F(ds)min-AF 

30 where AF is an arbitrary small value. When the relation shown in Mathematical 6 holds, the display position of the 
reproduced stereoscopic image is within the binocular fusing range; therefore, the fusing range verifier 18 supplies the 
current dx value as-is to the converging angle controller 19. When Ypmin satisfies the following relation 

35 [Mathematical 7] YPmin > F(ds)min +AF 

the display position of reproduced Ypmin is completely inside the binocular fusing range. In this case, the fusing range 
verifier 18 adds Adx to the current dx value and supplies the resulting new dx value to the converging angle controller 
1 9. However, when 0 to be described later (Figure 6) is close to 0, there occurs little change in the camera's optic axis 
40 angle 9 ; therefore, dx need not be updated. 

[0031] When Ypmin satisfies the following relation 



[Mathematical 8] YPmin < F(ds)min -AF 



the image reproduced at Ypmin is beyond the viewer's binocular fusing range. In this case, the current dx is reduced 
by Adx, and the result is taken as the new dx. If the value obtained by subtracting Ax from the new dx is smaller than 
dxO (>0, to be described later), then dxO is taken as the new dx. Calculation of dxO will now be described. The fusing 
range verifier calculates dx such that the y-coordinate of Ypmin agrees with F(ds)min in Figure 5, and takes the result 
so as dxO. 

[0032] The following equation is used to calculate dx, which is obtained by eliminating AS from equations in Math- 
ematical 5 and 2, substituting F(ds)min for Yp, and solving the result for dx. 

[Mathematical 9] dx o = OIA> 2F J dS ^ \ f '.^ J M ■ 
55 2We ds + F(ds)min*M-Axmm 

[0033] Using the thus obtained dx, the converging angle controller 1 9 controls the rotation controllers 4a and 4b to 
position the camera bodies 3a and 3b so that their optic axes are directed to the converging point shown in Figure 2 
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(a). Figure 6(a) shows how the camera control angle G is defined at this time. When 6 = 0, the optic axes of the two 
cameras are parallel to each other. In the above example, dx is updated in increments of Adx, but in the case of the 
condition shown by Mathematical 8, the value of dx_0 given by Mathematical 9 may be directly substituted into dx. 
[0034] In this case, control is performed so that Ypmin, the point in the object nearest to the viewer, becomes equal 
5 to F(ds)min, a point near the limit of the fusing range. At this time, if Ypmin is within the binocular fusing range under 
the condition that makes the optic axes of the camera bodies 3a and 3b parallel to each other, then the optic axes of 
the two cameras are maintained parallel to each other. 

[0035] As described above, according to the present embodiment, the parallax of captured images is calculated, and 
the optic axes of the stereoscopic pickup cameras are controlled so that the smallest parallax value can be reproduced 
10 within the fusing range of the viewer's eyes. When the images thus captured are presented for viewing, the viewer can 
fuse the reproduced stereoscopic images over the widest possible range. 

[0036] Further, in the first embodiment, the amount of change in dx is large in the vicinity of 9 = 0, as shown in Figure 

6(b); therefore, e may be made equal to 0 at dx_<» In this case, the initial value of dx is set as dx_<*>. 

[0037] Also t the curve shown in Figure 6(b) may be linearly approximated. Furthermore, in the conditional judgement 

is expressions shown in Mathematical 6 to 8, an offset Yp_off may be added to Ypmin, and the judgement condition may 
be shifted along the direction of the y-axis so that the position of Ypmin comes within the fusing range (shaded region). 
[0038] Figure 7 is a diagram showing the configuration of an image pickup section in a stereoscopic image pickup 
apparatus according to a second embodiment of the present invention. In the figure, the numerals 1 and 2 are lenses, 
3a and 3b are camera bodies, 4a and 4b are rotation controllers, 5 is a camera spacing controller, 1 7 is a depth display 

20 position calculator for calculating the spatial position of an object nearest to the viewer, and 19 is a converging angle 
controller for controlling the rotation controllers 4a and 4b so that the spatial position of the object nearest to the viewer 
comes within the fusing range of the viewer's eyes. The differences from the first embodiment are that the coordinates 
xO, yO of the nearest object are directly input to the depth display position calculator 17, and that a fusing range deter- 
miner is provided that calculates the value of dx to bring the object nearest to the viewer into the fusing range of the 

25 viewer's eyes. 

[0039] The operation of the thus constructed stereoscopic image pickup section of the second embodiment will be 
described below. 

[0040] In the shooting condition shown in Figure 2(a), the coordinates of the point in an object nearest to the cameras 
are set as N (xO, y0, zO). Prior to shooting the object, the operator measures this distance and inputs x0, yO to the 
30 depth display position calculator 17. These values may be input using a calculator keyboard or any other device that 
can enter numeric data. Based on these input values, the depth display position calculator 17 calculates the y<;oordi- 
nate of the display position of the point N in the stereoscopic image reproduction/display coordinate system shown in 
Figure 2(b). The calculation is made using the following equation. 



35 

[Mathematical 10] 



40 



w We • ds 
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In Mathematical 10, AS is determined in accordance with Mathematical 2, as in the first embodiment of the invention. 
First, Ypmin is calculated with the condition of dx = «, (the condition that sets the optic axes of the two cameras parallel 
to each other). Next, using the thus calculated Ypmin, the fusing range determiner 20 determines the converging point 
dx of the cameras, based on the following relation. If 

55 

[Mathematical 11] YPmin> F(ds)min 
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is satisfied, dx = <» (sufficiently large value) is taken; otherwise, the dx_0 given by Mathematical 9 is calculated, and 
the result is taken as the new dx. 

[0041] Based on the thus obtained dx, the converging angle controller 1 9 calculates the optic axis angle 0 of the two 
cameras. The angle 9 is the same as that shown in Figure 6. The rotation controllers 4a and 4b control the optic axes 

s of the respective cameras so that the thus determined angle G can be provided. 

[0042] As described above, according to the present embodiment, the position of the object nearest to the cameras 
is input, and the converging point dx is determined so that the object can be reproduced within the fusing range of the 
viewer's eyes, in accordance with which the optic axes of the stereoscopic image pickup cameras are controlled. When 
the images thus captured are presented for viewing, the viewer can fuse the reproduced stereoscopic images over the 

io widest possible range. 

[0043] In the second embodiment of the invention, the values xO, yO are input to the depth display position calculator 
17 manually by the operator, but these values may be input automatically by using a distance-measuring device such 
as a ultrasonic sensor. Also, xO may be fixed to a predetermined value, and the value of yO only may be input. 
[0044] Figure 8 is a diagram showing the configuration of an image pickup section in a stereoscopic image pickup 

is apparatus according to a third embodiment of the present invention. The block construction is the same as that shown 
in Figure 1 . In Figure 8, the numerals 1 and 2 are lenses, 3a and 3b are camera bodies, 4a and 4b are rotation controllers, 
5 is a camera spacing controller, 14a and 14b are signal processors each for converting the camera signal into the 
luminance signal, 1 5 is a parallax calculator for calculating horizontal parallax from left and right images, 1 6 is a parallax 
processor for detecting the smallest parallax, 17 is a depth display position calculator for calculating the spatial position 

20 of an object farthest from the viewer, and 18 is a fusing range verifier for verifying whether the spatial position of the 
object farthest from the viewer is within the fusing range of the viewer's eyes. 

[0045] Further, 19 is a converging angle controller for controlling the rotation controllers 4a, 4b so that the spatial 
position of the object farthest from the viewer comes within the fusing range of the viewer's eyes. 
[0046] The differences from the first embodiment are that the parallax processor 16 calculates the largest parallax 
25 value, that the depth display position calculator 1 7 calculates the spatial position of the object farthest from the viewer, 
and that the fusing range verifier 18 verifies whether the spatial position of the object farthest from the viewer is within 
the fusing range of the viewer's eyes. In the first embodiment, control was performed based on the smallest parallax 
value and the nearest object. 

[0047] The operation of the thus constructed stereoscopic image pickup section of the third embodiment will be 
30 described below. 

[0048] I n the first embodiment of the invention, the viewer's viewing distance was set greater than 71 cm, the binocular 
fusing range was from <» to F(ds)min, and ds was controlled so as to bring F(ds)min within the binocular fusing range 
of the viewer. When ds < 71 cm, the distance ranging from F(ds)max to F(ds)min is the binocular fusing range, as 
described earlier; if F(ds)min is to be brought within the fusing range, control can be performed in the same manner 
35 as described in the first embodiment, but in the third embodiment of the invention, control is performed in such a manner 
as to bring F(ds)max within the binocular fusing range. 

[0049] First, dx is initially set at <~ (sufficiently large value). Next, the parallax calculator 1 5 calculates a parallax map 
by using the image signals obtained from the camera bodies 3a, 3b and signal processors 14a, 14b, as in the first 
embodiment of the invention. The method of calculation is exactly the same as that employed in the first embodiment. 

40 Based on the parallax map, the parallax processor calculates the largest parallax value Axmax (the parallax of the 
object farthest from the cameras). Next, the depth display position calculator 17, using Mathematical 5, calculates the 
position at which the farthest object is to be reproduced for display. The coordinate system used here is the same as 
the stereoscopic image display coordinate system shown in Figure 2(b). In Figure 9, the viewer's binocular fusing range 
shown in Figure 3 is superimposed on the display model of Figure 2(b). The shaded portion represents the region 

45 where the viewer's binocular fusing can be attained. Here, F(ds)max is given by the upper curve in Figure 3. 

[0050] At this time, if the value of Ypmax and the value of the viewer's far point binocular fusing limit satisfy the relation 

[Mathematical 12] F(ds)max-AF ^ YPmax ^ F(ds)max 

50 

the display position of the reproduced stereoscopic image is within the binocular fusing range. In this case, the fusing 
range verifier 18 supplies the current dx value as-is to the converging angle controller 19. Further, if Ypmax satisfies 
the relation 

55 

[Mathematical 13] YPmax < F(ds) max -AF 
the display position of reproduced Ypmax is completely inside the binocular fusing range. In this case, the current dx 
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is reduced by Adx, and the result is taken as the new dx. If the value obtained by subtracting AF from the new dx is 
smaller than dx1 (>0 1 to be described hereinafter), then dx1 is taken as the new dx. Calculation of dx1 will now be 
described. The fusing range verifier 18 calculates dx such that the y-coordinate of Ypmax agrees with F(ds)max in 
Figure 5, and takes the result as dx1. 
s [0051] The following equation is used to calculate dx, which is obtained by eliminating AS from Mathematical 5 and 
2, making Yp= F(ds)min , and solving the result for dx, as in the first embodiment of the invention. 



rilt . * - 2F(ds)max-f-WcM 

[Mathematical 14] dx_1 = 2We ds ; F( ^ s)max , M , Axmax 



Further, if, as shown in Figure 9, Ypmax satisfies the relation 



[Mathematical 15] YPmax > F (ds) max + AF 

15 

the image reproduced at Ypmax is beyond the viewer's binocular fusing range. In this case, the fusing range verifier 
18 adds Adx to the current dx and supplies the result to the converging angle controller 19 as the new dx. However, 
when 6 to be described later (Figure 6) is close to 0, dx need not be updated since there occurs little change in the 
camera optic axis angle 9 described later. 

20 [0052] Using the thus obtained dx, the converging angle controller 19 controls the rotation controllers 4a and 4b to 
position the camera bodies 3a and 3b so that the camera optic axes are directed to the converging point shown in 
Figure 2(a). The camera control angle 6 here is defined in the same way as defined in the first embodiment of the 
invention with reference to Figure 6(a); that is, when G = 0, the optic axes of the two cameras are parallel to each other. 
In the above example, dx is updated in increments of Adx, but in the case of the condition shown by Mathematical 15, 

25 the value of dx_1 given by Mathematical 14 may be directly substituted into dx. 

[0053] In the above operation, control is performed so that Ypmax, the point in the object farthest from the viewer, 
becomes equal to F(ds)max, the far point limit of the fusing range. At this time, if Ypmax does not come within the 
binocular fusing range even under the condition that makes the optic axes of the camera bodies 3a and 3b parallel to 
each other, the optic axes of the two cameras are maintained parallel to each other. 

30 [0054] As described above, according to the present embodiment, the parallax of captured images is calculated, and 
the optic axes of the stereoscopic image pickup cameras are controlled so that the largest parallax value can be 
reproduced within the fusing range of the viewer's eyes. When the images thus captured are presented for viewing, 
the viewer can fuse the reproduced stereoscopic images over the widest possible range. 

[0055] In the first to third embodiments of the invention, noise may be introduced into the output of the parallax 
35 calculator 15, depending on the condition of the object being shot; therefore, the output of the parallax calculator 15 
may be passed through a time-domain low-pass filter to reduce the noise. Further, control of the two cameras need 
only be performed slowly; to prevent abrupt movements, Axmin output from the parallax processor 16, dx output from 
the fusing range verifier 18, or dx output from the fusing range determiner 20 may be low-pass filtered in the direction 
of time. 

40 [0056] Figure 1 6 is a diagram showing the configuration of a display section in a stereoscopic image pickup apparatus 
according to a fourth embodiment of the present invention. In Figure 16, the numeral 29 is a parallax calculator, 30 is 
an fixating point calculator, 31 is a parallax controller, and 32 is an image display. The operation of the thus constructed 
stereoscopic image display of the present embodiment will be described below. 

[0057] First, the parallax calculator 29 calculates a parallax map (a three-dimensional map) from left- and right-eye 
45 images. The same calculation method as used in the first embodiment will be used here, such as the correlation 
matching method that calculates the correlations of luminance patterns between the left and right images. That is, in 
Figure 4, consider a block window of n x n pixels (3x3 pixels in the figure) in the left image. Then, the same image 
as shown in the block window is located in the right image by using a window of the same size. In the displacement 
(Ax, Ay) between the left and right block positions, the horizontal displacement component Ax represents the binocular 
so parallax of the left and right images at the center coordinate of the block window. 

[0058] By horizontally shifting the reference block window in the left image in sequence across the entire screen, 
and finding the corresponding block position (representing the binocular parallax) in the right image for each shifted 
block position, a parallax map (showing depthwise distance at each position on the screen) can be obtained for the 
entire screen. For calculations, the same equation as used in the first embodiment is used, that is 

55 

[Mathematical 3] Ax = i, Ay = j for Min {Corr(i,j)} 
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20 



25 



where 

[Mathematical 4] 
5 nxn 

CorKi, j) = Y, GL(Xk,yk)-GR(Xk-i,y k -j) 

k=1 

10 

[0059] Based on the thus obtained parallax map (showing parallax Ax calculated at each coordinate position on the 
entire screen), the fixating point calculator 30 calculates an average value for the parallax over the entire screen or a 
weighted average value Axave with greater weight at the center of the screen. Figure 17 shows examples of positions 
in the screen and their associated weight coefficients. In Figure 17, the screen of X x Y size is simply divided into two 
1$ regions, a center region of X/2 x Y/2 size and the remaining region, and a weight coefficient K2 is assigned to the 
center region and K1 to the remaining region. Such regions may be in other shapes such as a circular shape, and the 
weight coefficient may be varied continuously from the periphery toward the center. The average value Axave is given by 



[Mathematical 16 J 



Axave = £ { K x AX ( i , j ) }yj X-Y(klxf- +K2x^)} 

'» J 



30 where 

35 

means summing about coordinate(i.j) on entire screen, and where K = K2 (region A), and K= K1 (region B). When K1 
= K2, a common average is taken over the entire screen. 

[0060] Using the average value Axave of the binocular parallax, the parallax controller 31 controls the horizontal 
read timing of the left and right images and moves the images in horizontal directions. Figure 18 shows a horizontal 

40 scanning period of the video signal. Points AL and AR indicate the same point in the same object in the left and right 
images, respectively. The binocular parallax AX at the point A, first mentioned, is shown in the figure. As shown in the 
figure, the right image is shifted by the average value Axave of the binocular parallax, in a direction that cancels the 
parallax (i.e., the horizontal read timing of the image is shifted by Axave). In this manner, the image portion having a 
binocular parallax of Axave is reproduced at the surface of the image display 32 (reproduced in the same position in 

45 the left and right images). Here, the entire images are moved horizontally ;as previously described, this is the same 
as changing LA and LB in Figure 15, and when the control amount of binocular parallax is small, the change goes 
relatively unnoticed to the viewer. If the value of Axave changes too rapidly causing jerky image movements on the 
display screen, the signal may be low-pass filtered so that the display screen can be controlled using slow movements 
only. 

so [0061] The fixating point calculator 30 calculates the largest parallax value, instead of an average value for the 
parallax over the entire screen. When the calculated value is positive, the parallax controller 31 controls the displace- 
ment between the left and right images on the screen of the image display 32 so that the maximum value of the 
displacement does not exceed the viewer's interocular distance (about 65 mm); this brings the displayed images always 
within the binocular fusing range, and ensures that the visual axes of the viewer's eyes are not forced to diverge. 

55 Further, the fixating point calculator 30 calculates the smallest parallax value, instead of an average value for the 
parallax over the entire screen. When the calculated value is negative, the parallax controller 31 controls the displace- 
ment between the left and right images on the screen of the image display 32 so that the minimum value of the dis- 
placement does not become smaller than a prescribed amount p; with this setting, the viewpoint of the viewer becomes 
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very near, eliminating a large disparity between the eye's focus information from the stereoscopic image display screen 
and the converging angle of the visual axes, and the left and right images can be controlled so that the viewer can 
easily fuse the images. 

[0062] Furthermore, if the amount of displayed image displacement is changed from Axave to Axave - a (a is an 
s arbitrary value), the portion of the parallax represented by Axave can be set to a prescribed binocular parallax value 
a. In this case, a = 0 indicates a position on the surface of the image display screen; this position moves forwardly or 
rearwardly of the image display screen as the value of a is varied. In this case, control is performed using one average 
binocular parallax value for one picture; on the other hand, in the case of a moving image, an average binocular parallax 
value may be obtained for each picture (one frame in an NTSC image), and the image display section may be controlled 
10 using a signal that is low-pass filtered in the direction of time with the average parallax values taken as time series 
data. In the first embodiment, only the right image is shifted; alternatively, the right image may be shifted by half the 
control amount of displacement while the left image is shifted in the opposite direction by the same amount. Also, 
instead of taking an average binocular parallax, a median (a median processing filter) may be used. 
[0063] In the present embodiment, the displayed images are controlled only in horizontal directions using the dis- 
15 placement Axave caused by horizontal parallax. In addition to this, if the image read position in the vertical direction 
is controlled by using Ayave based on the binocular parallax Ay calculated by the parallax calculator 29, stereoscopic 
images with reduced vertical parallax can be presented with enhanced visibility. (Vertical displacement between the 
left and right images would greatly interfere with the binocular fusing ability of the viewer.) 

[0064] As described above, according to the present embodiment, the stereoscopic image indicated by the average 
20 binocular parallax value (which may indicate the depthwise position of the center of the parallax map) can be controlled 
so that it will be reproduced at the surface of the display screen or at other desired position, thus enabling the viewer 
always to perceive the stereoscopic depth of the object over the widest possible range. 

[0065] Figure 1 9 is a diagram showing the configuration of a display section in a stereoscopic image pickup apparatus 
according to a fifth embodiment of the present invention. In Figure 19, the numeral 29 is a parallax calculator, 31 is a 

25 parallax controller, and 32 is an image display. These components are the same as those shown in Figure 16. The 
difference from the configuration shown in Figure 16 is that the configuration of Figure 1 9 has two additional functions: 
a lineof-sight sensor 33, a line-of-sight detector 34, and an fixating point evatuator 35 are provided, which together 
serve the function of measuring the line of vision (visual axis) that points to the position in a displayed image on which 
the viewer's eye is focused; and a control amount calculator 36 is provided to serve the function of controlling the 

30 amount of binocular parallax on the basis of the output of the parallax calculator and the information concerning the 
viewers' line of vision. 

[0066] The operation of the thus constructed stereoscopic image display section of the fifth embodiment will be 
described below. First, the parallax calculator 29 calculates a parallax map (a three-dimensional map) from left- and 
right-eye images, as in the fourth embodiment of the invention. The same calculation method as used in the fourth 

35 embodiment may be used for the above calculation; for example, the correlation matching method may be used which 
calculates the correlations of luminance patterns between the left and right images using the algorithm shown in Figure 
4. Based on the thus obtained parallax map (showing parallax Ax calculated at each coordinate position on the entire 
screen), and on the position (fixating point) the viewer is viewing on the display screen of the image display 32, the 
control amount calculator 36 calculates the amount of displacement in horizontal read timing between the left and right 

40 images input as signals to the parallax controller 31 . 

[0067] We will now describe a method of detecting the viewer's fixating point (the point on which the viewer's eye is 
focused). First, the direction of the visual axis of the viewer's eyeball is detected using the line-of-sight sensor 33 and 
line-of-sight detector 34. There are many detection methods, such as a limbus reflection method, a corneal reflection 
method, and a method involving monitoring the eye using a TV camera. Any of these methods may be used, but the 

45 following description deals specifically with the limbus reflection method. Figure 20 shows the operating principle of 
the line-of-sight sensor 33 in the limbus reflection method. Figure 20(a) is a front view of an eyeball, and (b) is a side 
view of the same. To detect a horizontal movement of the eyeball, the eyeball is illuminated with a weak infrared beam 
from an infrared LED, and the infrared beam reflected from the eyeball is measured with photodiodes. The photodiodes 
are directed to positions on both sides of the iris as shown by dashed lines, and the difference between the outputs of 

50 the photodiodes is calculated by an operational amplifier, to detect a horizontal movement of the eyeball. To detect a 
vertical movement of the eyeball, an infrared beam is directed to the lower part of the eyeball, as shown in Figure 20 
(b), and the reflected light is detected using a photodiode directed to the lower part of the iris. The thus obtained 
horizontal/vertical eyeball movement signals are converted by the line-of-sight detector 34 to a line-of-sight angle (line- 
of-sight signal). That is, the line of vision is measured in terms of horizontal angles, ax and ay, with respect to a pre- 

55 determined reference point (the optic axis of the viewer's eyeball is directed in the direction indicated by the angles). 
Next, based on the line-of-sight signal, the fixating point evaluator 35 calculates the position on the image display 32 
on. which the viewer's eye is focused. More specifically, when the distance from the viewer to the display screen of the 
image display 32 is denoted by L, the reference point is set at the center of the display screen, and a two-dimensional 
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coordinate system having its origin at the display screen center is defined, as shown in Figure 21 , then using the line- 
of-sight signal (ax, ay) the viewpoint F (x, y) defined in the two-dimensional screen coordinate system can be expressed 
as shown below. 

5 

[Mathematical 17] x = L x tan(ax) 
y =L X tan (ay) 

to Here, it is assumed that the viewer is positioned facing directly the center of the display screen, and that the display 
screen is not tilted with respect to the viewer. 

[0068] Using the thus obtained viewer's fixating point on the screen (the point in an image on which the viewer's 
fixating is focused), the fixating point evaluator 35 calculates the coordinates of the point on which the viewer's eye is 
rested. Figure 22(a) shows an example of time variation of the x component of the viewpoint F. When the viewer views 

is points x1 to x5 in sequence as shown, the viewer's eyeballs move very quickly as the viewpoint changes from x1 to 
x2 to x3, and so on. Such eye movement is generally referred to as saccade. When the viewer blinks his eye, it is not 
possible to measure the viewpoint since rapid movements of the eyelid cause abrupt changes in the waveform. The 
fixating point evaluator 35 removes such transient states and detects the state (attention state) in which the viewer's 
eye is rested on any of the points x1 to x5. More specifically, the fixating point evaluator 35 calculates the moving speed 

20 of the viewpoint, and determines that the viewer's eye is in the fixating state (marked * in Figure 22(b)) when the moving 
speed is less than a prescribed value (3, as shown in Figure 22(b). In the above calculation, only the x component of 
the viewpoint is used, but the fixating state may be detected by calculating a two-dimensional speed using both the x 
and y components. The time variation of the thus extracted viewer's fixating point is shown in Figure 22(c). 
[0069] Next, as shown in Figure 22(d), positional data of the fixating point is filtered through a zero-order hold, in- 

25 terpolating the time where no fixating point data is available. Thereafter, the time variation of the fixating point thus 
detected is smoothed by a low-lass filter. In this manner, the fixating point evaluator 35 detects the position in the 
displayed image on which the viewer's fixating is focused, and supplies the result as the fixating point signal to the 
control amount calculator 36. In the above example, the fixating point signal is created by detecting the lineof -sight 
direction of one eye. Alternatively, the line-of -sight directions of both eyes may be detected, and the fixating point signal 

30 may be calculated by using the average value of these data. 

[0070] Based on the thus obtained parallax map and fixating point signal, the control amount calculator 36 determines 
the control amount by which the parallax controller 32 controls the horizontal read timing of the left and right images. 
This operation will be described with reference to Figure 23. F designates the viewer's fixating point indicated by an 
fixating point signal at a given time. With this point as the center, the binocular parallaxes Ax shown on the parallax 

35 map are averaged over the range of M x M pixels. The result is taken as the average binocular parallax Axave which 
was explained in the fourth embodiment of the invention. Using this value, the parallax controller 31 controls the hor- 
izontal read timing of the left and right images, and moves the images horizontally. The control method is the same as 
that explained in the fourth embodiment with reference to Figure 18. That is, the right image is shifted by the average 
binocular parallax Axave (the horizontal read timing of the image is shifted by Axave). 

40 [0071] In the fifth embodiment of the invention, the control amount calculator 36 takes the average of the binocular 
parallaxes within the square area having the fixating point as its center. Alternatively, their weighted average may be 
taken with greater weight at its center, as in the example shown in Figure 17 in the fourth embodiment. 
[0072] Further, in the present embodiment, the lineK>f-sight signal is detected based only on the optic axis direction 
of the viewer. This method is effective when the viewer's head stays relatively still. In a freer viewing condition, a 

45 human's line of vision is given as a combined value of the direction in which the eyeball is pointed and the direction in 
which the head is moved. Methods of detecting the line of vision by combining the head and eyeball movements include 
a method of detecting the head of a viewer by a TV camera, and a method using a magnetic field generator in conjunction 
with a magnetic field detector mounted on the viewer's head, as disclosed in Japanese Patent Unexamined Publication 
No. 4-182226. Any of these methods may be employed. 

so [0073] As described above, according to the present embodiment, since the binocular parallax at the position in an 
image that the viewer is currently viewing can be set at 0 (the image is reproduced at the position of the display screen) 
or at a desired value, the stereoscopic image is always formed around the image position intended by the viewer, 
enabling the viewer to perceive the stereoscopic depth of an object over the widest possible range. 
[0074] Figure 24 is a diagram showing the configuration of a display section in a stereoscopic image pickup apparatus 

55 according to a sixth embodiment of the present invention. Figure 24(a) shows a stereoscopic image recording section, 
and (b) shows a stereoscopic image display section. In Figure 24, the numeral 29 is a parallax calculator, 31 is a 
parallax controller, 32 is an image display, and 36 is a control amount calculator. These components are the same as 
those used in the fifth embodiment of the invention (Figure 19). The difference from the fifth embodiment is that a 
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stereoscopic image pickup camera 37, a recorder 38, an fixating request pointer 39, and a reproducer 40 are added. 
[0075] The operation of the thus constructed display section of the sixth embodiment will be described below. First, 
in Figure 24(a), the operator shoots an object using the stereoscopic image pickup camera 37. The stereoscopic image 
pickup camera 37 can be a conventional stereoscopic camera consisting of two ordinary video cameras mounted side 

s by side on a universal head, or the stereoscopic image pickup section of the first embodiment of the invention. Captured 
images are recorded by the recorder (consisting of two conventional VTRs synchronized to each other). 
[0076] At this time, the operator inputs the position of the image on which the operator wishes the viewer to focus 
his fixating (an fixating request signal) by using the fixating request pointer 39. The fixating request signal is input using 
an ordinary pointing device, such as a mouse or tablet, and points to a given point in a captured image. This signal 

io varies with time. The fixating request signal at a given time t is designated by Fix (xt, yt), where xt and yt are two- 
dimensional coordinate values designating the position in a captured image pointed to by the pointing device. This 
fixating request signal is recorded together with the captured image by using the recorder 38 (VTRs). More specifically, 
the fixating request signal is sampled at a field frequency of 60 Hz, and inserted in the video signal during each vertical 
retrace interval (vertical blanking period). 

is [0077] In an alternative arrangement, the fixating request pointer 39 may be attached to an editing machine or a 
VTR so that the fixating request signal may be recorded when editing the recorded image. 

[0078] The thus recorded video signal and fixating request signal are processed in the stereoscopic image display 
section shown in Figure 24(b), and stereoscopic images are produced for display. The reproducer 40 (VTRs) reproduces 
the video signal and fixating request signaL The remainder of the operation is the same as that described in the fifth 

20 embodiment of the invention (Figure 1 9), except that the fixating point signal is replaced by the fixating request signal 
output from the reproducer 40. That is, based on the parallax map calculated by the parallax calculator 29 and on the 
fixating request signal reproduced by the reproducer 40, the control amount calculator 36 determines the control amount 
by which the parallax controller 32 controls the horizontal read timing of the left and right images. As shown in Figure 
23 in the fifth embodiment of the invention, the binocular parallaxes Ax shown on the parallax map are averaged over 

25 the M x M-pixel area centered around the image position on which the fixating request signal requests the viewer to 
focus his fixating. The result is taken as the average binocular parallax Axave which was explained in the fourth em- 
bodiment of the invention. Using this value, the parallax controller 31 controls the horizontal read timing of the left and 
right images, and moves the images horizontally. The control method is the same as that explained with reference to 
Figure 18. That is, the right image is shifted by the average binocular parallax Axave (the horizontal read timing of the 

30 image is shifted by Axave). 

[0079] In Figure 24, instead of reproducing the fixating request signal by the reproducer 40, the output of the fixating 
request pointer 39 may be fed directly to the control amount calculator 36 so that the viewer may input the fixating 
request signal while viewing the reproduced image. 

[0080] The fourth to sixth embodiment have dealt only with binocular stereoscopic images, but it will be recognized 
35 that, in a multinocular stereoscopic image pickup apparatus also, the same processing as described above can be 
applied to two images to be projected into the viewer's left and right eyes. In this case, however, since a plurality of 
left- and right-eye image pairs are presented for viewing at the same time, the same processing needs to be applied 
to each of the images. 

[0081] As described above, according to the present embodiment, by controlling the readout position of the stereo- 

40 scopic image using the fixating request signal, the visual program producer, the camera operator, or the viewer can 
bring the desired portion of the image onto the surface of the display screen or onto other desired position; as a result, 
the stereoscopic image is always formed with a wide fusing range around the image position intended by the viewer, 
enabling the viewer to perceive the stereoscopic depth of an object over the widest possible range. 
[0082] Figure 25 is a diagram showing the configuration of a stereoscopic image pickup apparatus according to a 

45 seventh embodiment of the present invention. In the figure, the numerals 1 and 2 are lenses, 3a and 3b are camera 
bodies, 4a and 4b are rotation controllers, 5 is a camera spacing controller, 14a and 14b are signal processors each 
for converting the camera signal into the luminance signal, 15 is a parallax calculator for calculating horizontal parallax 
from left and right images, 16 is a parallax processor for detecting the smallest parallax, 17 is a depth display position 
calculator for calculating the spatial position of an object nearest to the viewer, 1 8 is a fusing range verifier for verifying 

so whether the spatial position of the object nearest to the viewer is within the fusing range of the viewer's eyes, and 19 
is a converging angle controller for controlling the rotation controllers 4a, 4b so that the spatial position of the object 
nearest to the viewer comes within the fusing range of the viewer's eyes. These components are the same as those 
of the image pickup section (Figure 1) in the stereoscopic image pickup apparatus of the present invention. Further, 
the numeral 30 is an fixating point calculator, 31 is a parallax controller, and 32 is an image display. These components 

55 are the same as those of the display section (Figure 16) in the stereoscopic image pickup apparatus of the fourth 
embodiment of the invention. 

[0083] That is, in the seventh embodiment of the invention, both the image pickup section and the display section 
are controlled so that the presented stereoscopic image comes within the binocular fusing range of the viewer. The 
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operation of the thus constructed stereoscopic image pickup and display apparatus of the seventh embodiment will be 
described below. First, in shooting an object, control is performed so that dx in Figure 2(a) will have the largest possible 
value while satisfying the condition that enables all stereoscopic images to be reproduced within the fusional range. 
This operation is exactly the same as that described in the first embodiment of the invention. A brief description will 

5 be given here. To achieve the above condition, dx is initially set to «> (sufficiently large value). In Figure 25, the parallax 
calculator 15 calculates a parallax map (a three-dimensional map showing parallax Ax calculated at each coordinate 
position on the entire screen) from the left- and right-eye images output from the signal processors 14a and 14b. 
[0084] Based on this parallax map, the parallax processor 1 6 extracts the smallest parallax (the parallax of the object 
reproduced nearest to the viewer) from the entire screen, and takes the result as Axmin. Next, based on Axmin, the 

10 depth display position calculator 17 calculates the depth display position Ypmin of the stereoscopic image in the ster- 
eoscopic image presentation diagram shown in Figure 5, as follows. 



is 



20 



25 



30 



[Mathematical 5] 

2We-ds 



Ypmin = _ ^ ^ KL . 

(max) -M-AXmin»2-AS+2We 
(max) 

If this Ypmin is within the binocular fusing range, the current dx value is taken as is. If 

[Mathematical 7] YPmin > F (ds)min +AF 

a small value Adx is added to dx. If 

[Mathematical 8] YPmin < F(ds)min -AF 



a small value Adx is subtracted from dx. These calculations are exactly the same as those described in the first em- 
bodiment of the invention. Using the thus obtained dx value, the converging angle controller 19 controls the rotation 
35 controllers 4a and 4b. In this manner, control is performed so that Ypmin, the point in the object nearest to the viewer, 
becomes equal to F(ds)min, a point near the limit of the fusing range. 

[0085] Using the output of the parallax calculator 15, the fixating point calculator 30 obtains the average parallax or 
the weighted average parallax Axave for the entire screen, in accordance with which the parallax controller 31 scrolls 
the left and right images in such a manner as to eliminate the displacement of Axave between the left and right images. 

40 This processing is exactly the same as that described in the fourth embodiment of the invention. 

[0086] In the above operation, the converging point of the image pickup section and the displayed image position 
are controlled so that the stereoscopic image displayed by the image display 32 comes within the binocular fusing 
range of the viewer As a result, stereoscopic images with wide binocular fusing range can always be presented to the 
viewer, and eye strain during viewing can be reduced. 

45 [0087] In the above description of the seventh embodiment of the invention, the converging angle controller 1 9 and 
parallax controller 31 are operated simultaneously. Alternatively, to ensure the stability of the entire system operation, 
these two controllers may be operated in turn, or one controller may be operated after the operation of the other 
controller has stabilized. 

[0088] As described, according to the present invention, binocular parallax between images is detected, based on 
so which the optimum converging point of the cameras is calculated that enables the viewer to perceive the stereoscopic 
depth of aa object with the widest possible range, and the cameras are controlled so that their optic axes are directed 
to the converging point; further, the displayed image is automatically controlled so that the widest possible binocular 
fusing range can be provided for the displayed image, thereby achieving the presentation of stereoscopic images that 
help to reduce eye strain. 

55 



15 



EP0 641 132 B1 



Claims 

1. A stereoscopic image pickup apparatus comprising: 

an image pickup device (3a, 3b) for capturing an image of at least one object from a plurality of viewpoints; 

a converging angle moving mechanism (4a, 4b) for varying a converging angle of said image pickup device 
(3a, 3b) in a manner that changing the converging angle or parallel moving are executed until the at least one 
object falls within a fusional range of a viewer to minimise a 3D distortion when the image is displayed by a 
stereoscopic image display device; 

a signal processor (14a, 14b) for extracting image data from the image captured by said image pickup device 
(3a, 3b); 

a parallax calculator (1 5) utilising blockmatching technique between said plural viewpoint images for calculat- 
ing a parallax of the image by using an output of said signal processor (14a, 14b); 

a parallax processor (16) for detecting the smallest value out of the outputs from said parallax calculator (15), 
to determine the parallax of the nearest of said at least one object; 

a depth display position calculator (17) for calculating from an output of said parallax processor (16) a depth- 
wise position of an object to be reproduced at the nearest point when the captured image is displayed by a 
stereoscopic image display device; 

a fusing range verifier (18) for verifying on the basis of an output of said depth display position calculator (17) 
whether said depthwise position is within the fusional range of a viewer when the viewer views the reproduced 
image; and 

a converging angle controller (19) for controlling said converging angle moving mechanism (4a, 4b) on the 
basis of an output of said fusing range verifier so that the object at the nearest point is positioned within the 
fusional range of the viewer. 

2. The stereoscopic image pickup apparatus according to claim 1, wherein 

the parallax processor (16) detects the largest value from outputs from said parallax calculator (15), to deter- 
mine the parallax of the farthest object; 

the depth display position calculator (17) calculates from an output of said parallax processor (16) the depth- 
wise position of an object to be reproduced at the farthest point when the captured image is displayed by the 
stereoscopic image display device; . 

said fusing range verifier (18) verifies whether said farthest point is within the fusional range of the viewer 
when the viewer views the reproduced image; and v 

said converging angle controller (19) controls said converging angle moving mechanism on the basis of an 
output of said fusing range verifier (18) so that the object at the farthest point is positioned within the fusional 
range of the viewer 

3. A stereoscopic image pickup apparatus comprising: 

an image pickup device (3a, 3b) for capturing an image of at least one object from a plurality of viewpoints; 

a converging angle moving mechanism (4a, 4b) for varying the converging angle of said image pickup device 
(3a, 3b) in a manner that changing the converging angle or parallel moving are executed until the at least one 
object falls within a fusional range of a viewer to minimise a 3D distortion when the image is displayed by a 
stereoscopic image display device; 

a depth display position calculator (17) for calculating, by using positional data of the nearest of the at least 
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one object, a depthwise position of an object to be reproduced at the nearest point when the captured image 
is displayed by a stereoscopic image display device; 

a fusing range determiner (20) for calculating from an output of said depth display position calculator (17) a 
condition that achieves a reproduction of an image of the object at said nearest point within the binocular 
fusing range of a viewer; and 

a converging angle controller (19) for controlling said converging angle moving mechanism (4a, 4b) on the 
basis of an output of the fusing range determiner (20). 

4. The stereoscopic image pickup apparatus according to claim 3, wherein: 

said depth display position calculator (17) calculates, by using positional data of the farthest of the at least 
one object, a depthwise position of an object to be reproduced at the farthest point when the captured image 
is displayed by a stereoscopic image display device; 

said fusing range determiner (20) determines whether the farthest point is within the fusional range of the 
viewer when the viewer views the reproduced image; and 

the converging angle controller (4a, 4b) controls said converging angle moving mechanism on the basis of an 
output of the fusing range determiner (20) so that said farthest point comes within the fusional range of the 
viewer viewing the reproduced image. 

Patentanspruche 

1. Stereoskopische Bildabtastvorrichtung, umfassend: 

ein Bildabtastgerat (3a, 3b) zum Erfassen eines Bildes wenigstens eines Objektes von einer Vielzahl von 
Standpunkten; 

einen Konvergenzwinkelbewegungsmechanismus (4a, 4b) zum Verandern eines Konvergenzwinkels <Jes 
Bildabtastgerates (3a, 3b) in einer Weise, da3 das Andern des Konvergenzwinkels oder paralleles Bewegen 
ausgefuhrt werden, bis das wenigstens eine Objekt in den Verschmelzungsbereich eines Betrachters fallt, um 
eine 3D-Verzerrung zu minimieren, wenn das Bild von einem stereoskopischen Bildanzeigegerat angezeigt 
wird; 

einen Signalprozessor (14a, 14b) zum Gewinnen von Bilddaten aus dem von dem Bildabtastgerat (3a, 3b) 
erfaBten Bild; 

eine Parallaxenberechnungseinrichtung (15), die eine Blockanpassungstechnik zwischen den mehreren 
Standpunktbildern verwendet, zum Berechnen einer Parallaxe des Biides unter Verwendung einer Ausgabe 
des Signalprozessors (14a, 14b); 

einen Parallaxenprozessor (16) zum Ermitteln des kleinsten Wertes aus den Ausgaben von der Parallaxen- 
berechnungseinrichtung (15), um die Parallaxe des nachstgelegenen der wenigstens einen Objekte zu be- 
stimmen; 

eineTiefenanzeigepositionsberechnungseinrichtung (17) zum Berechnen einer tiefenmaBigen Position eines 
an dem nachstgelegenen Punkt zu reproduzierenden Objektes aus einer Ausgabe des Parallaxenprozessors 
(16), wenn das erfaBte Bild von einem stereoskopischen Bildanzeigegerat angezeigt wird; 

eine VerschmelzungsbereichsuberprOf ungseinrichtung (1 8) zum Uberprufen auf der Grundlage einer Ausgabe 
der Tiefenanzeigepositionsberechnungseinrichtung (17), ob die tiefenmaBige Position innerhalb des Ver- 
schmelzungsbereichs eines Betrachters liegt, wenn der Betrachter das reproduzierte Bild betrachtet; und 

eine Konvergenzwinkelsteuerung (19) zum Steuern des Konvergenzwinkelbewegungsmechanismus (4a, 4b) 
auf der Grundlage einer Ausgabe der VerschmelzungsbereichsuberprOf ungseinrichtung, so daB das Objekt 
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an dem nachstgelegenen Punkt innerhalb des Verschmelzungsbereichs des Betrachters positioniert wird. 
Die stereoskopische Bildabtastvorrichtung nach Anspruch 1, wobei: 

der Parallaxenprozessor (1 6) den groRten Wert aus den Ausgaben von der Parallaxenberechnungseinrichtung 
(15) ermittelt, urn die Parallaxe des weitestgelegenen Objektes zu bestimmen; 

die Tiefenanzeigepositionsberechnungseinrichtung (17) die tiefenmaGige Position eines an dem weitestgele- 
genen Punkt zu reproduzierenden Objektes aus einer Ausgabe des Parallaxenprozessors (16) berechnet, 
wenn das erfaGte Bild von dem stereoskopischen Bildanzeigegerat angezeigt wird; 

die Verschmelzungsbereichuberprufungseinrichtung (18) uberpruft, ob der weitestgelegene Punkt innerhalb 
des Verschmelzungsbereichs des Betrachters liegt, wenn der Betrachterdas reproduzierte Bild betrachtet; und 

die Konvergenzwinkelsteuerung (19) den Konvergenzwinkelbewegungsmechanismus auf der Grundlage ei- 
ner Ausgabe der Verschmelzungsbereichuberprufungseinrichtung (18) steuert, so daG das Objekt an dem 
weitestgelegenen Punkt innerhalb des Verschmelzungsbereichs des Betrachters positioniert wird. 

Stereoskopische Bildabtastvorrichtung, umfassend: 

ein Bildabtastgerat (3a, 3b) zum Erfassen eines Bildes wenigstens eines Objektes von einer Vielzahl von 
Standpunkten; 

einen Konvergenzwinkelbewegungsmechanismus (4a, 4b) zum Verandern des Konvergenzwinkels des Bild- 
abtastgerates (3a, 3b) in einer Weise, daG das Andern des Konvergenzwinkels oder paralleles Bewegen aus- 
gefuhrt werden, bis das wenigstens eine Objekt in einen Verschmelzungsbereich des Betrachters fallt, urn 
eine 3D-Verzerrung zu mtnimieren, wenn das Bild von einem stereoskopischen Bildanzeigegerat angezeigt 
wird; 

eineTiefenanzeigepositionsberechnungseinrichtung (17) zum Berechnen einer tiefenmaGigen Position eines 
an dem nachstgelegenen Punkt zu reproduzierenden Objektes durch Verwendung von Positionsdaten des 
nachstgelegenen der wenigstens einen Objekte, wenn das erfaGte Bild von einem stereoskopischen Bildan- 
zeigegerat angezeigt wird; 

eine Verschmelzungsbererchsfeststellungseinrichtung (20) zum Berechnen einer Bedingung aus einer Aus- 
gabe der Tiefenanzeigepositionsberechnungseinrichtung (17), die eine Reproduktion eines Bildes des Objek- 
tes an dem nachstgelegenen Punkt innerhalb des binokularen Verschmelzungsbereichs eines Betrachters 
erzielt; und 

eine Konvergenzwinkelsteuerung (19) zum Steuem des Konvergenzwinkelbewegungsmechanismus (4a, 4b) 
auf der Grundlage einer Ausgabe der Verschmelzungsbereichsfeststellungseinrichtung (20). 

Die stereoskopische Bildabtastvorrichtung nach Anspruch 3, wobei: 

die Tiefenanzeigepositionsberechnungseinrichtung (17) eine tiefenmaGige Position eines an dem weitestge- 
legenen Punkt zu reproduzierenden Objektes unter Verwendung von Positionsdaten des weitestgelegenen 
der wenigstens einen Objekte berechnet, wenn das erfaGte Biid von einem stereoskopischen Bildanzeigegerat 
angezeigt wird; 

die Verschmelzungsbereichsfeststellungseinrichtung (20) feststellt, ob der weitestgelegene Punkt innerhalb 
des Verschmelzungsbereichs des Betrachters liegt, wenn der Betrachter das reproduzierte Bild betrachtet; und 

die Konvergenzwinkelsteuerung (4a, 4b) den Konvergenzwinkelbewegungsmechanismus auf der Grundlage 
einer Ausgabe der Verschmelzungsbereichsf eststellungseinrichtung (20) steuert, so daG der weitestgelegene 
Punkt in den Verschmelzungsbereich des das reproduzierte Biid betrachtenden Betrachters kommt. 
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Revendications 

1. Dispositif de prise d'image stereoscopique comprenant : 

un dispositif de prise d'image (3a, 3b) pour saisir une image d'au moins un objet a partir d'une pluralite de 
points de visee ; 

un mecanisme de deplacement d'angle de convergence (4a, 4b) pour faire varier un angle de convergence 
dudit dispositif de prise d'image (3a, 3b) de maniere que soit un changement de Tangle de convergence, soit 
un deplacement parallele soit execute jusqu'a ce qu'au moins un objet se trouve a I'interieur d'une plage de 
fusion d'un observateur pour minimiser une distorsion tridimensionneile lorsqu'une image est visualisee par 
un dispositif d'affichage d'image stereoscopique ; 

un processeur de signaux (14a, 14b) pour extraire les donnees representatives de I'image a partir de I'image 
saisie par ledit dispositif de prise d'image (3a, 3b) ; 

un calculateur de parallaxe (15) utilisant une technique d'adaptation de blocs entre ladite multitude d'images 
de point de visee pour calculer un parallaxe de I'image en utilisant une sortie dudit processeur de signaux 
(14a, 14b) ; 

un processeur de parallaxe (16) pour detecter la valeur la plus petite parmi les sorties dudit calculateur de 

parallaxe (15) pour determiner le parallaxe dudit objet le plus proche dudit au moins un objet ; 

un calculateur de position d'affichage de profondeur (17) pour calculer a partir d'une sortie dudit processeur 

de parallaxe (16) une position dans le sens de la profondeur d'un objet qui doit etre reproduit au point le plus 

proche lorsque i'image saisie est visualisee par un dispositif d'affichage d'image stereoscopique ; 

un verificateur de plage de fusion (18) pour verifier, sur la base d'une sortie dudit calculateur de position 

d'affichage de profondeur (17), si ladite position dans le sens de la profondeur est a I'interieur de la plage de 

fusion d'un observateur lorsque I'observateur observe I'image reproduce ; et 

un controleur d'angle de convergence (19) pour commander ledit mecanisme de deplacement d'angle de 
convergence (4a, 4b), sur la base d'une sortie dudit verificateur de plage de fusion, de sorte que I'objet au 
point le plus proche est positionne a I'interieur de la plage de fusion de I'observateur. 

2. Dispositif de prise d'image stereoscopique selon la revendication 1, dans lequel 

le processeur de parallaxe (16) detecte la valeur la plus grande a partir des sorties dudit calculateur de pa- 
rallaxe (15) pour determiner le parallaxe de I'objet le plus loin ; 

te calculateur de position d'affichage de profondeur (17) calcule a partir d'une sortie dudit processeur de pa- 
rallaxe (16) la position dans le sens de la profondeur d'un objet qui doit etre reproduit au point le plus loin 
lorsque I'image saisie est visualisee par le dispositif d'affichage d'image stereoscopique ; 
ledit verificateur de plage de fusion (18) verifie si le point le plus loin est a I'interieur de la plage de fusion de 
I'observateur lorsque I'observateur observe I'image reproduite ; et 

ledit controleur d'angle de convergence (1 9) commande ledit mecanisme de deplacement d'angle de conver- 
gence, sur la base d'une sortie dudit verificateur de plage de fusion (18) de sorte que I'objet au point le plus 
loin est positionn6 a I'interieur de la plage de fusion de I'observateur. 

3. Dispositif de prise d'image stereoscopique comprenant : 

un dispositif de prise d'image (3a, 3b) pour saisir une image d'au moins un objet a partir d'une pluralite de 
points de visee ; 

un mecanisme de defacement d'angle de convergence (4a, 4b) pour faire varier Tangle de convergence dudit 
dispositif de prise d'image (3a, 3b) d'une maniere telle que le changement de Tangle de convergence ou le 
deplacement parallele sont executes jusqu'a ce qu'au moins un objet se trouve a I'interieur d'une plage de 
fusion d'un observateur pour minimiser une distorsion tridimensionneile lorsque i'image est visualised par le 
dispositif d'affichage d'image stereoscopique ; 

un calculateur de position d'affichage de profondeur (17) pour calculer, en utilisant les donnees positionne lies 
du point le plus proche d'au moins un objet, une position dans te sens de la profondeur d'un objet qui doit etre 
reproduit au point le plus proche lorsque I'image saisie est visualisee par un dispositif d'affichage d'image 
stereoscopique ; 

un determineur de plage de fusion (20) pour calculer a partir d'une sortie dudit calculateur de position d'affi- 
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chage de profondeur (17) une condition qui obtient une reproduction de I'image de I'objet audit point le plus 
proche a I'interieur de la plage de fusion binoculaire d'un observateur; et 

un controleur d'angle de convergence (19) pour commander ledit mecanisme de deplacement d'angle de 
convergence (4a, 4b) sur la base d'une sortie dudit determineur de plage de fusion (20). 

Dispositif de prise d'image stereoscopique selon la revendication 3, dans lequel : 

ledit calculateur de position d'affichage de profondeur (17) calcule, en utilisant les donnees positionnelles du 
point le plus loin d'au moins un objet, une position dans le sens de la profondeur d'un objet qui doit etre 
reproduit au point le plus loin lorsque I'image saisie est visualised par un dispositif d'affichage d'image 
stereoscopique ; 

ledit determineur de plage de fusion (20) determine si le point le plus loin est a I'interieur de la plage de fusion 
de I'observateur lorsque I'observateur observe I'image reproduce ; et 

le controleur d'angle de convergence (4a, 4b) commande ledit mecanisme de deplacement d'angle de con- 
vergence, sur la base d'une sortie dudit determineur de plage de fusion (20), de sorte que ledit point le plus 
loin arrive a I'interieur de la plage de fusion de I'observateur observant I'image reproduite. 
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Fig. 5 




25 



EP0 641 132 B1 



X 




CD 




CO 
« 



26 



EP0 641 132 B1 













E- 




< 








^> 




O 


< 




_) 




cu 




in 




hi 




Q 


o 




J— 1 






H 


M 


PU 


CO 


w 


o 


Q 


a. 



o 

CM" 



w 




o 








< 


W 


OS 


55 




h-t 


O 




rz; 


OS 






CO 










Q 



CD 



X 











o 








< 




o 


a: 






M 


— 1 


O 


t-J 


0^ 


o 


W 




> 








o 


o 


o 


a 




feuo 



27 



EP0 641 132 B1 



ID. 









o 




£- 




< 










>* 


U 


< 






< 


Cu 


o 


CO 




M 




Q 


Q 






*T* 




c~* 




n . 


CO 




<— > 


Q 


a. 






X 




< 






05 




o 


< 


CO 




CO 




en 


< 


o 


EC 


o 




os 




a. 



u 




a 




z 




< 




05 


at 


w 


CD 


t— 1 


Z 


Li- 


t-H 


^-H 


CO 


OS 


ZD 


U 




> 



o 



Q 
O 
CQ 



O 
Z 
n 
U 
< 

a. 

CO 

< 

OS 

u 
< 




00 



X 



w 




.J 




o 




z 




< 




o 




z 




M 




a 




05 


© 




cr 


> 




z 


z 


o 


o 


o 


o 



CD 





O 




05 




05 


o 




O 


CO 


< 


CO 


*J CO 


05 


•J CO 


< td. 


w 


< fc] 


z o 


AM 


z o 


o o 


a o 


►H 05 


a 


►H 05 


CO D- 




CO o- 



J2> 



en 



05 

o 

05 
H 
Z 

o 

o 



oo 




28 



EP0 641 132 B1 



Fig. 9 
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